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ABSTRACT — This paper outlines an extension of the Krausz method for nonlinear system identifica- 
tion in which the Poisson random impulse trains are used as the test inputs and the system is described by 
the Volterra functional expansion. The extension concerns improvement of accuracy in the nonlinear 
system identification that the test inputs are not perfectly random, in other words, their spectra are 
non-white. A formula for the improvement of the system kernels in the functional expansion are given 
under such input condition. This approach is illustrated and evaluated with analyses of the responses in 
the crayfish brain activated by the pseudo-random impulse train stimuli. When the input spectral 
property was apparently deviated from white, the accuracy of the system identification was efficiently 
improved through the present procedure. Even if the input impulse trains are not perfectly random, 
they can be used as the test inputs in order to identify nonlinear systems through the simple extension of 
the Krusz method. 


INTRODUCTION 

White noise analysis has recently been a power- 
ful tool for investigation of information processing 
in living neuron networks. In the Lee-Schetzen 
method for identification of non-linear systems, 
the Wiener kernels for the functional expansion 
can be evaluated by the multiple cross correlations 
between “the white and Gaussian noise input” and 
the resultant system output [1]. In another 
approach of nonlinear system identification, 
Krausz has derived a similar method in which the 
Volterra kernels are obtained from the multiple 
cross-correlations between “the Poisson random 
impulse train input” and the resultant system out- 
put, because the ideal Poisson process is the white 
noise [2]. The Poisson random impulse train 
inputs have been practically used in the linear and 
nonlinear analyses of the nerve signal transimis- 
sions in the crayfish brain [3-5]. If we are released 
from the strict condition that the test inputs must 
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be the white noise in the Krausz method, spon- 
taneous spike trains which are observed in some 
central nervous system and not so perfectly ran- 
dom could be employed as the test inputs for 
nonlinear analyses of neuron networks in it. 

An extension of the Wiener filter for linear 
system identification to nonlinear system analysis 
had been investigated first by lick [6]. This 
approach brings about the same result as the 
extension of the Lee-Schetzen method using the 
Gaussian white noise inputs to the use of the 
Gaussian non-white inputs in identification of non- 
linear systems. Several authors have described 
almost same approaches as the Tick’s one [7-10]. 
In this study, an extension of the Krausz method to 
the use of non-white inputs was tried with the 
pseudo-random impulse train inputs in the analysis 
of the nerve spike train transmissions in the 
crayfish brain. 

MATERIAL AND METHODS 

The experimental date used in the present com- 
putations were mainly gathered in the brain (cere- 
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bral ganglion) and partly in the abdominal gang- 
lion of the crayfish ( Procambarus clarkii). The 
stimuli to the brain were applied to the nerve 
bundle in the optic peduncle. At the same time, 
the nerve spike trains of the brain outputs were 
recorded extracellularly from the large descending 
axons in the circumesophageal connective and 
intracellularly in the brain [4, 5, 11]. The input- 
output data of the abdominal ganglion cells were 
obtained by stimulation of axon bundles in the 
interganglionic connective and extracellular re- 
cording from the axons in the ganglionic roots. 
The nerve cells in which the experimental data 
were obtained were named by means of alpha- 
numeric specification (Table 1). The alphabet in 
each cell name indicates whether the cell is in the 
brain (B-) or in the abdominal ganglion (A-). The 
stimuli were generated with a random pulse gener- 
ator [3], then the pulse trains which had various 
degree of “non-white” spectral properties were 
picked up for the present purpose (Fig. 1 and 
Table 1). 

Theory 

In the Krausz method that the Poisson random 
impulse train input is used as the test input x(t ), the 
system output y(t ) is given by 

y(t) = Go+Gi + G2~\~ ( 1 ) 

using the functionals 


in the brackest [2, 5]. 

The extension of the Wiener filter theory to the 
nonlinear system identification, or the extension of 
the Lee-Schetzen method to the use of the non- 
white Gaussian input, has been given in the fre- 
quency domain by several authors [6-10]. 
Although the test input must be Gaussian and 
white in the Lee-Schetzen method, only Gaussian- 
ity of it is necessary in the extended method. The 
bases of this extension are that the odd-order 
moments of the Gaussian process always vanish [9] 
and that the even-order moments of it are decom- 
posed into the product of its second order moment 
[ 12 ]- 

The higher order moments of the Poisson pro- 
cess can be also decomposed into the product of 
the second order moment of it [2]. Therefore, it is 
considered that, when the spectrum of the input is 
non- white and also not so far from white, the same 
procedure as the Gaussian input case can be used 
in order to improve the accuracy in nonlinear 
system identification by the Krausz method, as 
approximation. In this study, the improvement of 
the system kernels was performed in the time 
domain by the reasons mentioned in the discus- 
sion. The time domain version of the extension is 
given as follows 



•M<*l— T l)-M T n—Tn)dTi...dT n , 


Gq~ ho 


G i- 
G 2 = 


h\{r)x{t— r)dr 

Tz)x(t—Ti) 
•X(t— T2)dz\dT2 , 


( 2 ) 


where ho is the mean value of y(r), and h\ and h 2 
are the first and second order system kernels, 
respectively. The n-th order system kernel can be 
obtained by the multiple cross-correlation 

Mn, .... *-)=-d_£ I WO -EG*] (3) 

n\A k= \ 

■x(t-Ti)...x(t-r, i)| , 

where A is the mean rate of the input impulses and 
E | | indicates the expectation of the content 


where </> xy is the n-th order cross-correlation func- 
tion between the input and the output, h n is the 
improved n-th order system kernel and (f > x , the 
auto-correlation function of the input. 

Computation 

Signal intensity was set to be 1 in a bin (10 msec 
width) where the nerve impulse (or the stimulus 
pulse) was present and, otherwise, to be 0. First, 
the unimproved Volterra kernels up to the third 
order were calculated with the equation-(3) and 
the improved kernels of the same orders were 
derived with the equation-(4). The computation 
with the original Krausz method was described 
detailedly in the previous paper [5]. The equation- 
(4) can be transformed by discretization into a 
system of linear algebraic equations, whose coef- 
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ficients construct the N-fold block Toeplitz matrix. 
Several methods were tried in order to solve the 
systems of the linear equations corresponding to 
the present input-output data. These systems of 
the linear equations can be solved by use of the 
regular structure of the nested block Toeplitz 
matrix and also any of the usual iterative methods 
[13, 14]. 

Next, the unimproved model output r(/) and the 
improved one i(/) were computed respectively 
with the euqations-(l) and -(2) from each input- 
output pair used in the above kernel computation. 
Then the both model outputs were compared with 
the output of the same pair, i.e. the brain output 
impulse train (Fig. 3). Last of all, the MSE (mean 
square error) between the unimproved model out- 
put z(t) and the brain output v(/) 

e=E\(y(t)-z(t)) 2 \ (6) 

and the MSE between the improved model output 
and the same brain output as used above 

e=E\(yit)-z(t)) 2 \ (7) 


were calculated. E j | denotes time average. 
Then improvement of the model output was evalu- 
ated by percentage of (e- e) in e for each input- 
output pair. The power spectra of the test inputs 
were calculated from the normalized auto- 
correlation functions of them through the Fourier 
transform. And the deviation of each input spec- 
trum was evaluated also by means of the MSE 
between it and the spectrum of the white noise of 
unit power (Fig. 1). 

Computations in this work had been begun with 
a mini-computor HITAC 10-11 and were carried 
out on several personal computors (Hitachi B- 16/ 
B-32 series and others). 

RESULTS 

The input power spectra of the input-output 
data used in this study are not flat, in other words, 
non-white at a glance; but their deviations are not 
so large as over ±3 db (Fig. 1). Accordingly, it 
seems that these input-output data are suitable to 
improve the system kernels with the present exten- 






Fig. 1. Power spectra of the random pulse trains used as the test inputs for identification of the impulse signal 
transfer systems in the crayfish central nervous systems. The broken line in each graph of the spectrum indicates 
the unit power level of the white noise. 
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sion of the Krausz method, i.e. the equation-(4). 
The MSEs of the deviations of the input spectra, 
the MSEs of the abrrations of the model outputs 
which are calculated with the system kernels up to 
the second orderd and up to the third order and 
the perentages of the model output improvement 
are given in Table 1. The MSE of the third order 
model was usually smaller than the MSE of the 
second order one for each input-output pair in 
both unimproved and improved cases. And the 
improvement percentage of the third order model 
was mostly larger than the second order one and 
had a tendency to increase depending upon aug- 
mentation of the deviation of the input power 
spectrum (Fig. 2). 

The upper half of Fig. 3 illustrates the third 
order model outputs before (a) and after (b) 
kernel improvement for the data of the B04-cell, in 
which relatively large improvement, 9.76%, is 
obtained. Some large aberrations could be recog- 
nized at several places in the unimproved model 
output (Fig. 3a) and they decreased noticeabley in 
the improved one (Fig. 3b). The lower half of Fig. 
3 shows the effect of improvement of the kernels 
up to the third order for the data of the B03-cell. 
In this case, the MSE of the third order model was 
extremely small, the percentage of improvement 
was also small, 2.56%, and the large aberrations 
could not be discriminated over the whole range of 
the model outputs before (Fig. 3c) and after (Fig. 
3d) improvement of the system kernels. 
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Fig. 2. Relationship between the deviations (MSEs) of 
the test input spectra from the unit power level of 
the white noise and the improvements (%) of the 
model outputs of the second order (open marks) and 
the third order (filled marks) for the responses of the 
brain cells (circles) and the abdominal ganglion cells 
(squares) (see Table 1). 


Table 1. The MSEs of the deviations of the input power spectra, the MSEs of the aberrations in the 
unimproved and improved model outputs of the second order and the third order and the 
improvement percentages of the model outputs. Asterisks indicate the exceptional cases (see Fig. 
4). Units of the MSEs are not comparable between the inputs and the model outputs. 


Cell 

names 

MSEs of 
input 
spectra 

MSEs of 2nd order outputs 

improvement 
of 2nd order 
outputs (%) 

MSEs of 3rd order outputs 

improvement 
of 3rd order 
outputs (%) 

unimproved 

improved 

unimproved 

improved 

B03 

0.49 

19.87 

19.79 

0.40 

9.00 

8.77 

2.65 

B39 

0.87 

29.65 

28.73 

3.10 

21.42 

20.63 

3.69 

B38 

1.01 

30.42 

29.81 

2.01 

24.70 

23.40 

5.26 

A13 

1.03 

26.93 

25.38 

5.76 

14.83 

13.37 

9.84 

A50 

1.04 

18.14 

17.15 

5.46* 

11.45 

13.49 

-17.81* 

B40 

1.19 

25.19 

24.13 

4.20 

15.71 

13.63 

13.24 

B02 

1.45 

28.80 

26.36 

8.47* 

38.73 

21.67 

44.05* 

B04 

1.50 

23.87 

22.62 

5.24 

11.67 

10.53 

9.76 
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Fig. 3. Comparisons between the unimproved model output (a) and the improved one (b) for the response of the 
B04-cell and between the unimproved (c) and improved model outputs for the response of the B03-cell. In each 
record, lower solid trace represents the stimulus pulse train, the upper dotted trace represents the brain output 
impulse train and the upper solid trace shows the model outputs calculated with the unimproved (a and c) and 
improved (b and d) kernels up to the third order. Horizontal scale: 200 msec. Vertical scale: Unit impulse height 
(inapplicable to stimuli). 


The two exceptional cases were observed in the 
present study (asterisks in Table 1). For the B02- 
cell, the MSE of the unimproved third order model 
output was larger than the MSE of the unimproved 
second order one, but the MSE of the third order 
model output drastically decreased after improve- 
ment of the third order system kernel (Table 1, 
and Fig. 4a, 4b). It may be noticed that the data of 
the B02-cell has been obtained in the brain im- 
mediately after application and removal of picroto- 
xin (10“ 5 M). 

The exception of another type occurred in an 
abdominal ganglion cell (A50); the improvement 
of the third order kernel caused remarkable de- 
terioration of the model output (Table 1, and Fig. 
4c. 4d). In this case, the MSE of the second order 
model output was very small beforehand with 
construction of the third order one (Table 1). It 


was difficult to predict the behaviours of the third 
order model outputs from the features of the 
second order models in these exceptional cases. 

Generally speaking, it can be concluded the 
larger the deviation of the input power spectrum, 
the more the model output of any order is im- 
proved (Fig. 2). 

DISCUSSION 

Although the equation for the extension of the 
Wiener filter to the nonlinear case can be derived 
in the time domain, most authors have given the 
equation in the frequency domain [7-9]. They 
should naturally consider that it can be solved 
practically by use of the fast Fourier transform. 
However, the latest author has pointed out that 
the computations of the high-order spectral density 
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Fig. 4. The exceptional cases that the drastic changes occurred in the model outputs after improvement of the third 
order system kernels, shown by asterisks in Table 1. The model outputs before (a) and after (b) improvement of 
the system kernels up to the third order for the response of the B02-cell. The model outputs before (c) and after 
(d) improvement of the system kernels up to the third order for the response of the A50-cell. Horizontal scale: 
200 msec. Vertical scale: Unit impulse height. 


functions are usually very expensive and time- 
consuming [10]. In addition, moving average 
techniques as the data window and the spectral 
window are indispensable for stable computation 
of the frequency domain version of the equation- 
(4) by use of the discrete data time series and the 
discrete Fourier transform [8]. Consequently, very 
high sampling rate must be employed in order to 
reproduce sharply each impulse in the model out- 
put after application of such moving average tech- 
niques. Therefore, the processing of very large 
number of the data points is not avoidable in 
identification of the impulse output nonlinear sys- 
tems in the frequency domain. However the 
Krausz method can be used without any moving 
average techniques in order to identify the impulse 
input-output nonlinear systems [5]. Thus the ex- 
tension of the Krausz method was tried in the time 
domain. 


In spite of the non-Gaussian property of the 
pseudo-random impulse train inputs used in this 
study, the present results indicate that the im- 
provement of the system kernels with the equa- 
tion-^) can reduce efficiently the identification 
errors resulted from the non-white spectral proper- 
ty of the present test inputs. And the kernel 
estimation admits of further improvement when 
effects of the odd-order moments of the test input 
can be taken into account in computation of the 
system kernels, because the odd-order moments of 
the pseudo-random impulse train do not vanish 
usually. But, the effects of the corrections of the 
second order system kernels by the third order 
moments of the inputs were very small in the 
present computation; for instance, about 0.2% for 
the input-output pair of the B02-cell. In any case, 
it is considered that spontaneously evoking nerve 
spike trains in some living neuron network can be 
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used as the test inputs for nonlinear analyses of 
nerve signal transfers in it through the present 
extension of the Krausz method, even if they are 
not so highly random. 
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